Image processing method, image display method, image processing device, image display device, image processing program, and image display program

ABSTRACT

A non perfusion area is detected from a fundus image. A fundus image is acquired, a first non perfusion area in a first region of the fundus is extracted from the fundus image, and a second non perfusion area in a second region of the fundus is extracted from the fundus image.

TECHNICAL FIELD

The present invention relates to an image processing method, an imagedisplay method, an image processing device, an image display device, animage processing program, and an image display program.

BACKGROUND ART

Patent Document 1 discloses analysis of a tomographic image of a fundusto extract a region where an abnormality has developed. The ability toconfirm an abnormality by analyzing a fundus image is desirable.

RELATED ART DOCUMENTS Patent Documents

Patent Document 1: US Patent Application Publication No. 2015/0366452

SUMMARY OF INVENTION

A first aspect of technology disclosed herein is an image processingmethod including: a step of acquiring a fundus image; a step ofextracting a first non perfusion area in a first region of a fundus fromthe fundus image; and a step of extracting a second non perfusion areain a second region of the fundus from the fundus image.

A second aspect of technology disclosed herein is an image displaymethod including: a step of acquiring a fundus image and informationrelating to a first non perfusion area in a first region of a fundusextracted from the fundus image, and a second non perfusion area in asecond region of the fundus; and a step of displaying at least one outof the first non perfusion area or the second non perfusion areasuperimposed on the fundus image.

A third aspect of technology disclosed herein is an image processingprogram to cause a computer to execute the image processing method ofthe first aspect.

A fourth aspect of technology disclosed herein is an image processingprogram to cause a computer to execute the image display method of thesecond aspect.

A fifth aspect of technology disclosed herein is an image processingdevice including: a fundus image acquisition section configured toacquire a fundus image; a first non perfusion area extraction sectionconfigured to extract from the fundus image a first non perfusion areain a first region of a fundus; and a second non perfusion areaextraction section configured to extract from the fundus image a secondnon perfusion area in a second region of the fundus.

A sixth aspect of technology disclosed herein is an image display deviceincluding; an acquisition section configured to acquire a fundus image,information relating to a first non perfusion area in a first region ofa fundus extracted from the fundus image, and information relating to asecond non perfusion area in a second region of the fundus; and adisplay section configured to display at least one out of the first nonperfusion area or the second non perfusion area superimposed on thefundus image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating an ophthalmic system according toan exemplary embodiment.

FIG. 2 is a schematic configuration diagram illustrating an overallconfiguration of an ophthalmic device according to an exemplaryembodiment.

FIG. 3 is a block diagram illustrating a configuration of an electricalsystem of a management server according to an exemplary embodiment.

FIG. 4 is a block diagram illustrating a configuration of an electricalsystem of an image viewer according to an exemplary embodiment.

FIG. 5 is a block diagram illustrating functionality of a CPU of amanagement server according to an exemplary embodiment.

FIG. 6 is a block diagram illustrating functionality of a CPU of animage viewer according to an exemplary embodiment.

FIG. 7 is a flowchart for an image processing program executed by amanagement server according to an exemplary embodiment.

FIG. 8 is a diagram illustrating a fundus region in a fundus imageaccording to an exemplary embodiment.

FIG. 9 is a flowchart illustrating a flow of processing to detect nonperfusion areas at a fundus posterior pole portion according to anexemplary embodiment.

FIG. 10 are explanatory diagrams illustrating processing to detect nonperfusion areas at a fundus posterior pole portion according to anexemplary embodiment. FIG. 10A illustrates a primary candidate, FIG. 10Billustrates exclusion of a primary candidate, and FIG. 10C illustratesidentified non perfusion areas at a fundus posterior pole portion.

FIG. 11 is a flowchart illustrating a flow of processing to detect nonperfusion areas at a fundus peripheral portion according to an exemplaryembodiment.

FIG. 12 is a diagram illustrating a display screen displayed on an imageviewer display according to an exemplary embodiment.

FIG. 13 is a diagram illustrating a display screen displayed on an imageviewer display according to an exemplary embodiment.

FIG. 14 is a flowchart illustrating a flow of processing according to afirst modified example.

FIG. 15 is a flowchart illustrating a flow of processing according to asecond modified example.

FIG. 16 is a flowchart illustrating a flow of processing according to aneighth modified example.

DESCRIPTION OF EMBODIMENTS

Detailed explanation follows regarding an exemplary embodiment of thepresent invention, with reference to the drawings.

Configuration of an ophthalmic system 100 will now be explained withreference to FIG. 1. As illustrated in FIG. 1, the ophthalmic system 100includes an ophthalmic device 110, a laser treatment device 120, amanagement server device (referred to hereafter as “management server”)140, and an image display device (referred to hereafter as “imageviewer”) 150.

The ophthalmic device 110 acquires fundus images and tomographic images.The laser treatment device 120 is a device to support treatment ofpathological lesions in an examined eye 12 of a patient. An example ofthe laser treatment device 120 is a medical apparatus used to suppressprogression of a pathological lesion on the optical fundus of a patient,such as a laser photocoagulator that illuminates laser light to causephotocoagulation of the illuminated site. The laser treatment device 120transmits information relating to treatment performed on the examinedeye 12 to the management server 140. For example, when a particular siteon the retina of the examined eye 12 is treated, the position of thisparticular site, the treatment time, and the treatment method aretransmitted to the management server 140 as treatment information.

The management server 140 stores plural fundus images obtained byimaging the fundi of plural patients using the ophthalmic device 110,and stores these in association with patient IDs. The management server140 also detects non perfusion areas (NPAs) in a specified fundus image.The image viewer 150 displays images corresponding to the results ofanalysis of the fundus images by the management server 140, such asestimated non perfusion areas (NPAs).

The non perfusion areas (NPAs) referred to herein are regions of thefundus where there is no, or very little, blood flow due to occlusionsof the retinal capillary bed, for example. They may also correspond toregions where retinal ischemia has occurred due to perfusion injury.

The ophthalmic device 110, the laser treatment device 120, themanagement server 140, and the image viewer 150 are coupled togetherover a network 160.

Although, as described above, the ophthalmic system 100 illustrated inFIG. 1 includes the laser treatment device 120, the technology disclosedherein is not limited thereto. For example, the laser treatment device120 of the ophthalmic system 100 may be swapped for a measurementinstrument such as a field of view measurement instrument for measuringthe visual field of a patient, or an eye axial length measurementinstrument for measuring the eye axial length, this being the length ofthe examined eye 12 along an eye axial direction. Moreover, suchadditional measurement instruments may also be connected over thenetwork 130.

The management server 140 is an example of an “image processing device”of technology disclosed herein. The image viewer 150 is an example of an“image display device” of technology disclosed herein.

For ease of explanation, hereinafter “scanning laser ophthalmoscope”will be abbreviated to SLO, and “optical coherence tomography” will beabbreviated to OCT.

Explanation follows regarding configuration of the ophthalmic device110, with reference to FIG. 2.

In cases in which the ophthalmic device 110 is installed on a horizontalplane with a horizontal direction taken as an X direction, a directionperpendicular to the horizontal plane is taken as a Y direction, and adirection connecting the center of the pupil at the anterior segment ofthe examined eye 12 and the center of the eyeball is taken as a Zdirection. The X direction, the Y direction, and the Z direction arethus mutually orthogonal directions.

The ophthalmic device 110 includes an imaging device 14 and a controldevice 16. The imaging device 14 is provided with an SLO unit 18 and anOCT unit 20, and acquires a fundus image of the fundus of the examinedeye 12. Two-dimensional fundus images that have been acquired by the SLOunit 18 are referred to hereafter as SLO images. Tomographic images,face-on images (en-face images) and the like of the retina created basedon OCT data acquired by the OCT unit 20 are referred to hereafter as OCTimages.

The control device 16 includes a computer provided with a CentralProcessing Unit (CPU) 16A, Random Access Memory (RAM) 16B, Read-OnlyMemory (ROM) 16C, and an input/output (I/O) port 16D.

The control device 16 is provided with an input/display device 16Ecoupled to the CPU 16A through the I/O port 16D. The input/displaydevice 16E includes a graphical user interface to display images of theexamined eye 12 and to receive various instructions from a user. Anexample of the graphical user interface is a touch panel display.

The control device 16 is provided with an image processing device 17coupled to the I/O port 16D. The image processing device 17 generatesimages of the examined eye 12 based on data acquired by the imagingdevice 14. Note that the control device 16 is coupled to the network 130through a communication interface, not illustrated in the drawings.

Although the control device 16 of the ophthalmic device 110 is providedwith the input/display device 16E as illustrated in FIG. 2, thetechnology disclosed herein is not limited thereto. For example, aconfiguration may adopted in which the control device 16 of theophthalmic device 110 is not provided with the input/display device 16E,and instead a separate input/display device is provided that isphysically independent of the ophthalmic device 110. In such cases, thedisplay device is provided with an image processing processor unit thatoperates under the control of a display control section 204 of the CPU16A in the control device 16. Such an image processing processor unitmay display SLO images and the like based on an image signal output inresponse to an instruction from the display control section 204.

The imaging device 14 operates under the control of an image capturecontrol section 202 of the control device 16. The imaging device 14includes the SLO unit 18, an image capture optical system 19, and theOCT unit 20. The image capture optical system 19 includes a firstoptical scanner 22, a second optical scanner 24, and a wide-angleoptical system 30.

The first optical scanner 22 scans light emitted from the SLO unit 18two dimensionally in the X direction and the Y direction. The secondoptical scanner 24 scans light emitted from the OCT unit 20 twodimensionally in the X direction and the Y direction. As long as thefirst optical scanner 22 and the second optical scanner 24 are opticalelements capable of polarizing light beams, they may be configured byany out of, for example, polygon mirrors, mirror galvanometers, or thelike. A combination thereof may also be employed.

The wide-angle optical system 30 includes an objective optical system(not illustrated in FIG. 2) provided with a common optical system 28,and a combining section 26 that combines light from the SLO unit 18 withlight from the OCT unit 20.

The objective optical system of the common optical system 28 may be areflection optical system employing a concave mirror such as anelliptical mirror, a diffraction optical system employing a wide-anglelens, or may be a reflection-diffraction optical system employing acombination of a concave mirror and a lens. Employing a wide-angleoptical system that utilizes an elliptical mirror, wide-angle lens, orthe like enables imaging to be performed of not only a central portionof the fundus (fundus posterior pole portion), but also of the retina ata peripheral portion of the fundus.

For a system including an elliptical mirror, a configuration may beadopted that utilizes an elliptical mirror system as disclosed inInternational Publication (WO) Nos. 2016/103484 or 2016/103489. Thedisclosures of WO Nos. 2016/103484 and 2016/103489 are incorporated intheir entirety by reference herein.

Observation of the fundus over a wide field of view (FOV) 12A isimplemented by employing the wide-angle optical system 30. The FOV 12Arefers to a range capable of being imaged by the imaging device 14. TheFOV 12A may be expressed as a view angle. In the present exemplaryembodiment the viewing angle may be defined in terms of an internalillumination angle and an external illumination angle. The externalillumination angle is the angle of illumination by a light beam shonefrom the ophthalmic device 110 toward the examined eye 12, and is anangle of illumination defined with respect to a pupil 27. The internalillumination angle is the angle of illumination of a light beam shoneonto the fundus F, and is an angle of illumination defined with respectto an eyeball center O. A correspondence relationship exists between theexternal illumination angle and the internal illumination angle. Forexample, an external illumination angle of 120° is equivalent to aninternal illumination angle of approximately 160°. The internalillumination angle in the present exemplary embodiment is 200°.

SLO fundus images obtained by imaging over a view angle having aninternal illumination angle of 160° or greater are referred to asUWF-SLO fundus images. UWF is an abbreviation of ultra-wide field.

An SLO system is realized by the control device 16, the SLO unit 18, andthe image capture optical system 19 as illustrated in FIG. 2. The SLOsystem is provided with the wide-angle optical system 30, enablingfundus imaging over the wide FOV 12A.

The SLO unit 18 is provided with a blue (B) light source 40, a green (G)light source 42, a red (R) light source 44, an infrared (for examplenear infrared) (IR) light source 46, and optical systems 48, 50, 52, 54,56 to guide the light from the light sources 40, 42, 44, 46 onto asingle optical path using transmission or reflection. The opticalsystems 48, 50, 56 are configured by mirrors, and the optical systems52, 54 are configured by beam splitters. B light is reflected by theoptical system 48, is transmitted through the optical system 50, and isreflected by the optical system 54. G light is reflected by the opticalsystems 50, 54, R light is transmitted through the optical systems 52,54, and IR light is reflected by the optical systems 52, 56. Therespective colors of light are thereby guided onto a single opticalpath.

The SLO unit 18 is configured so as to be capable of switching betweenthe light source or the combination of light sources employed whenemitting laser light of different wavelengths, such as in a mode inwhich R light and G light are emitted, a mode in which infrared light isemitted, etc. Although the example in FIG. 2 includes three lightsources, i.e. the G light source 42, the R light source 44, and the IRlight source 46, the technology disclosed herein is not limited thereto.For example, the SLO unit 18 may, furthermore, also include a blue (B)light source or a white light source, in a configuration in which hat isemitted according to various modes, such as a mode in which G light, Rlight, and B light are emitted or a mode in which white light is emittedalone.

Light introduced to the image capture optical system 19 from the SLOunit 18 is scanned in the X direction and the Y direction by the firstoptical scanner 22. The scanning light passes through the wide-angleoptical system 30 and the pupil 27 and is shone onto the fundus.Reflected light that has been reflected by the fundus passes through thewide-angle optical system 30 and the first optical scanner 22 and isintroduced into the SLO unit 18.

The SLO unit 18 is provided with a beam splitter 64 and a beam splitter58. From out of the light coming from the posterior eye portion (fundus)of the examined eye 12, the B light therein is reflected by the beamsplitter 64 and light other than the B light is transmitted through thebeam splitter 64. From out of the light transmitted through the beamsplitter 64, the G light therein is reflected by the beam splitter 58and light other than the G light is transmitted through the beamsplitter 58. The SLO unit 18 is further provided with a beam splitter 60that, from out of the light transmitted through the beam splitter 58,reflects R light therein and transmits light other than the R light. TheSLO unit 18 is further provided with a beam splitter 62 that reflects IRlight from out of the light transmitted through the beam splitter 60.The SLO unit 18 is further provided with a B light detector 70 to detectthe B light reflected by the beam splitter 64, a G light detector 72 todetect the G light reflected by the beam splitter 58, an R lightdetector 74 to detect the R light reflected by the beam splitter 60, andan IR light detector 76 to detect the IR light reflected by the beamsplitter 62.

Of the light that has passed through the wide-angle optical system 30and the first optical scanner 22 and been introduced into the SLO unit18 (i.e. reflected light that has been reflected by the fundus), the Blight therein is reflected by the beam splitter 64 and photo-detected bythe B light detector 70, and the G light therein is reflected by thebeam splitter 58 and photo-detected by the G light detector 72. R lightof this introduced light is transmitted through the beam splitter 58,reflected by the beam splitter 60, and photo-detected by the R lightdetector 74. IR light of this introduced light is transmitted throughthe beam splitters 58, 60, reflected by the beam splitter 62, andphoto-detected by the IR light detector 76. The image processing device17 that operates under the control of the CPU 16A employs detectionsignals from the B light detector 70, the G light detector 72, the Rlight detector 74, and the IR light detector 76 to generate UWF-SLOimages.

These UWF-SLO images include a UWF-SLO image (G fundus image) obtainedby imaging the fundus in green, and a UWF-SLO image (R fundus image)obtained by imaging the fundus in red. The UWF-SLO images furtherinclude a UWF-SLO image (B fundus image) obtained by imaging the fundusin blue, and a UWF-SLO image (IR fundus image) obtained by imaging thefundus in IR.

The control device 16 may control the light sources 40, 42, 44 so as toemit light at the same time as each other. The G fundus image, the Rfundus image, and the B fundus image may be obtained at mutuallycorresponding positions by imaging the fundus of the examined eye 12using B light, G light, and R light at the same time. An RGB colorfundus image may be obtained from the G fundus image, the R fundusimage, and the B fundus image. The control device 16 may also controlthe light sources 42, 44 so as to emit light at the same time as eachother. The G fundus image and the R fundus image are obtained atmutually corresponding positions by imaging the fundus of the examinedeye 12 using G light and R light at the same time in this manner. An RGcolor fundus image may be obtained from the G fundus image and the Rfundus image.

The UWF-SLO images may further include a UWF-SLO fluoroscopy imageobtained by fluoroscopy using a contrast agent.

Image data for the B fundus image, the G fundus image, the R fundusimage, the IR fundus image, the RGB color fundus image, the RG colorfundus image, and the UWF-SLO fluoroscopy image are sent from theophthalmic device 110 to the management server 140 through anon-illustrated communication IF.

An OCT system is realized by the control device 16, the OCT unit 20, andthe image capture optical system 19 illustrated in FIG. 2. The OCTsystem is provided with the wide-angle optical system 30. This enablesfundus imaging to be performed over the wide FOV 12A similarly to whenimaging the SLO fundus images as described above. The OCT unit 20includes a light source 20A, a sensor (detector) 20B, a first lightcoupler 20C, a reference optical system 20D, a collimator lens 20E, anda second light coupler 20F.

Light emitted from the light source 20A is split by the first lightcoupler 20C. After one part of the split light has been collimated bythe collimator lens 20E into parallel light, to serve as measurementlight, the parallel light is introduced into the image capture opticalsystem 19. The measurement light is scanned in the X direction and the Ydirection by the second optical scanner 24. The scanned light is shoneonto the fundus through the wide-angle optical system 30 and the pupil27. Measurement light that has been reflected by the fundus passesthrough the wide-angle optical system 30 and the second optical scanner24 so as to be introduced into the OCT unit 20. The measurement lightthen passes through the collimator lens 20E and the first light coupler20C before being introduced to the second light coupler 20F.

The remainder of the light emitted from the light source 20A and splitby the first light coupler 20C is introduced into the reference opticalsystem 20D as reference light, and is introduced to the second lightcoupler 20F through the reference optical system 20D.

The respective lights introduced to the second light coupler 20F, namelythe measurement light reflected by the fundus and the reference light,interfere with each other in the second light coupler 20F so as togenerate interference light. The interference light is photo-detected bythe sensor 20B. The image processing device 17 operating under thecontrol of an image processing control section 206 generates OCT images,such as tomographic images and en-face images, based on OCT datadetected by the sensor 20B.

Note that the OCT unit 20 is able to obtain OCT data for OCT images,which are tomographic images of the examined eye 12. Examples of OCTimages include: a one-dimensional OCT image that is an A-scan imageobtained by performing what is referred to as an A-scan using theophthalmic device 110; a two-dimensional OCT image that is a B-scanimage obtained by performing what is referred to as a B-scan using theophthalmic device 110; and a three-dimensional OCT image that is aC-scan image obtained by performing what is referred to as a C-scanusing the ophthalmic device 110.

OCT fundus images obtained by imaging over a view angle having aninternal illumination angle of 160° or greater are referred to asUWF-OCT images.

Image data of the UWF-OCT images is sent from the ophthalmic device 110to the management server 140 through the non-illustrated communicationIF and stored in a storage device 254.

Note that although in the present exemplary embodiment an example isgiven in which the light source 20A is a swept-source OCT (SS-OCT), thelight source 20A may be from various OCT systems, such as from aspectral-domain OCT (SD-OCT) or a time-domain OCT (TD-OCT) system.

Explanation follows regarding a configuration of an electrical system ofthe management server 140, with reference to FIG. 3. As illustrated inFIG. 3, the management server 140 is provided with a computer unit 252.The computer unit 252 includes a CPU 262, RAM 266, ROM 264, and aninput/output (I/O) port 268. A storage device 254, a display 256, amouse 255M, a keyboard 255K, and a communication interface (I/F) 258 arecoupled to the input/output (I/O) port 268. The storage device 254 is,for example, configured by non-volatile memory. The input/output (I/O)port 268 is coupled to the network 130 through the communicationinterface (I/F) 258. The management server 140 is thus capable ofcommunicating with the ophthalmic device 110, the laser treatment device120, and the image viewer 150.

The management server 140 stores various data received from theophthalmic device 110 and the laser treatment device 120 in the storagedevice 254.

Explanation follows regarding a configuration of an electrical system ofthe image viewer 150, with reference to FIG. 4. As illustrated in FIG.4, the image viewer 150 is provided with a computer unit 152. Thecomputer unit 152 includes a CPU 162, RAM 166, ROM 164, and aninput/output (I/O) port 168. A storage device 154, a display 156, amouse 155M, a keyboard 155K, and a communication interface (FF) 158 arecoupled to the input/output (I/O) port 168. The storage device 154 is,for example, configured by non-volatile memory. The input/output (I/O)port 168 is coupled to the network 130 through the communicationinterface (I/F) 158. The image viewer 150 is thus capable ofcommunicating with the ophthalmic device 110 and the management server140.

Explanation follows regarding various functions implemented by the CPU262 of the management server 140 executing an image processing program,with reference to FIG. 5. As illustrated in FIG. 5, the image processingprogram includes a display control function, an image processing controlfunction, and a processing function. The CPU 262 functions as thedisplay control section 204, the image processing control section 206,and a processing section 208 illustrated in FIG. 5 by the CPU 262executing the image processing program that includes these functions.

Next, explanation follows regarding various functions implemented by theCPU 162 of the image viewer 150 executing the image processing program,with reference to FIG. 6. As illustrated in FIG. 6, the image processingprogram includes a display control function, an image processing controlfunction, and a processing function. The CPU 162 functions as a displaycontrol section 104, an image processing control section 106, and aprocessing section 108 illustrated in FIG. 6 by the CPU 162 executingthe image processing program that includes these functions.

The image processing control section 206 is an example of a “fundusimage acquisition section”, a “first non perfusion area extractionsection”, and a “second non perfusion area extraction section” oftechnology disclosed herein.

The image processing control section 106 is an example of an“acquisition section” of technology disclosed herein, and the display156 is an example of a “display section” of technology disclosed herein.

Next, detailed explanation follows regarding image processing by themanagement server 140, with reference to FIG. 7. The image processingillustrated in the flowchart of FIG. 7 is implemented by the CPU 262 ofthe management server 140 executing the image processing program.

The image processing illustrated in FIG. 7 is an example of an imageprocessing method of technology disclosed herein. Moreover, the displayprocessing to display images obtained by the image processingillustrated in FIG. 7 is an example of an image display method oftechnology disclosed herein.

Note that various data (for example, image data of UWF-SLO images, andimage data of UWF-OCT images) obtained by executing processing such asUWF-SLO image processing and UWF-OCT image processing is saved in thestorage device 254.

In the present exemplary embodiment, an example of a UWF-SLO imageemployed in NPA detection is a UWF-SLO image obtained by fluoresceinangiography (FA). Such images are referred to hereafter as FA images.Capillaries of the retina are imaged at high resolution in FA images,and so FA images are well-suited to non perfusion area detection. Theimage data of such FA images is sent from the ophthalmic device 110 tothe management server 140 via a non-illustrated communication IF, and isstored in the storage device 254.

Note that UWF-SLO images obtained by imaging using indocyanine green(ICG) as a contrast agent (obtained by indocyanine green angiography)(hereafter referred to as IA images) may also be employed. The bloodvessels of the fundus appear white in FA images and IA images.

First, at step S102 in FIG. 7, the image processing control section 206function realized by the management server 140 acquires the FA imagefrom the storage device 254. Note that the FA images are imaged usingthe ophthalmic device 110 and then stored in the storage device 254.

Next, at step S104, the image processing control section 206 detects thefundus region of the examined eye 12 in the acquired FA image. At stepS104, the fundus region of the examined eye 12 is detected by removingsites peripheral to the examined eye 12, such as the eyelashes andeyelids of the patient, and regions where elements configuring theophthalmic device 110 intrude into the image, and extracting theremaining region as the fundus region. This processing to extract thefundus region may be implemented by image processing configured by acombination of known processing, such as binarization processing andmorphological processing. FIG. 8 illustrates an image of an extractedfundus region after the processing of step S104 has been performed. Thebroken line in FIG. 8 serves to schematically illustrate the edge of thefundus region, and no such broken line is actually present in the imageafter performing the processing of step S104.

Next the image processing control section 206 detects non perfusionareas (NPAs) in the detected fundus region.

Since, for example, there are differences between the distribution ofretinal blood vessels at the fundus posterior pole portion and in aperipheral portion of the fundus peripheral to the fundus posterior poleportion, in the present exemplary embodiment image processing ispreferably executed according to the site within the fundus region ofthe examined eye 12. Accordingly, in the present exemplary embodiment,image processing is performed to visually classify the non perfusionareas (NPAs) of the fundus region into non perfusion areas (NPAs) at thefundus posterior pole portion (hereafter referred to as posterior poleportion NPAs) and non perfusion areas in the fundus peripheral portion(hereafter referred to as UWF-NPAs). A posterior pole portion NPA is asearch target in a fundus region where retinal blood vessels are groupedclosely together, and a UWF-NPA is a search target in the fundusperipheral region. Thus in the present example, the FA image issubjected to processing optimized for detection of posterior poleportion NPAs (the processing of FIG. 9 or FIG. 16, described later), andprocessing optimized for detection of UWF-NPAs (the processing of FIG.11, described later).

Specifically, at step S106 the image processing control section 206performs detection for posterior pole portion NPAs in the fundus regionof the examined eye 12, and at step S108 the image processing controlsection 206 performs image processing on the detected posterior poleportion NPAs to pick out any misdetected posterior pole portion NPAsamong the detected posterior pole portion NPAs. In addition, the imageprocessing control section 206 performs UWF-NPA image processing at stepS110 to detect for UWF-NPAs in the fundus region of the examined eye 12.Note that a misdetected posterior pole portion NPA is a region that hasa low likelihood of being an actual posterior pole portion NPA despitehaving been detected as a posterior pole portion NPA.

The fundus posterior pole portion of the examined eye 12 is an exampleof a “first region” of technology disclosed herein, and the posteriorpole portion NPA is an example of a “first non perfusion area” oftechnology disclosed herein. Moreover, the fundus peripheral portion, atthe periphery of the fundus posterior pole portion of the examined eye12, is an example of a “second region” of technology disclosed herein,and the UWF-NPA is an example of a “second non perfusion area” oftechnology disclosed herein. Moreover, a misdetected posterior poleportion NPA, this being a region having a low likelihood of being anactual posterior pole portion NPA, is an example of a “third nonperfusion area” of technology disclosed herein.

Note that either out of step S106 followed by step S108, or step S110,may be executed first, or execution thereof may be simultaneous. Afterthe processing of both step S106 followed by step S108 and step S110 hasbeen completed, the image processing control section 206 executes imageprocessing at step S112 to generate a display screen. As will bedescribed in detail later, the display screen thus generated is adisplay screen of the FA image, with outlines of posterior pole portionNPAs displayed superimposed on the FA image so as to enable thepositions of the posterior pole portion NPAs to be easily recognized,and with outlines of UWF-NPAs displayed superimposed on the FA image soas to enable the positions of the UWF-NPA to be easily recognized. Atstep S114, the processing section 208 sends a display image of thegenerated display screen to the image viewer 150.

Next, explanation follows regarding image processing for posterior poleportion NPAs, with reference to FIG. 9 and FIG. 10.

As illustrated in FIG. 9, at step S304, the image processing controlsection 206 performs image emphasis processing on the acquired FA imageto emphasize the vascular portions thereof. This processing isprocessing to make the blood vessels, including capillary blood vessels,more prominent in order to estimate posterior pole portion NPAs withgood precision.

The image emphasis processing may employ various methods, such asemphasis processing in which an image histogram is subjected tohistogram averaging or contrast limited adaptive histogram equalization(CLAHE), or alternatively contrast conversion processing based ongradation conversion, frequency emphasis processing for a particularfrequency band employing an unsharp mask or the like, deconvolutionprocessing employing a Weiner filter or the like, morphology processingto emphasize the shape of the vascular portions, or the like. Preferablyhistogram averaging or adaptive histogram equalization is employedtherefor. The blood vessels are emphasized as a result of the imageemphasis processing.

Next the image processing control section 206 estimates plural posteriorpole portion NPAs from the FA image in which the blood vessels have beenemphasized. More specifically, at step S306, the image processingcontrol section 206 selects primary candidates for posterior poleportion NPAs. More specifically, the image processing control section206 extracts plural pixels of a first darkness or darker from the FAimage in which the blood vessels have been emphasized, and selects asprimary candidates for the posterior pole portion NPAs a single orplural regions having a surface area of a prescribed surface area orgreater of contiguous pixels of the first darkness or darker.

The pixels of the first darkness or darker referred to here are pixelshaving a pixel value of a first prescribed value or lower. For example,brightness values expressing lightness may be employed as the pixelvalues. Alternatively, values expressing at least one out of saturationor hue may be employed as the pixel values instead of brightness values,or in addition to brightness values.

Next, the image processing control section 206 executes the imageprocessing of step S308 and step S310.

At step S308, from out of the single or plural primary candidates forposterior pole portion NPAs, the image processing control section 206selects only dark candidates based on an average value of the respectivepixel values in each of the candidate regions. Specifically, the imageprocessing control section 206 calculates an average value of the pixelvalues in each of the regions corresponding to the single or pluralprimary candidates for posterior pole portion NPAs, and selects as adark region a single or plural candidates having a calculated averagevalue smaller than a second prescribed value. The second prescribedvalue is a prescribed value smaller than the first prescribed value.Namely, only candidates corresponding to dark regions having a darknessthat is a second darkness darker than the first darkness, or darker(i.e. candidates having a prescribed average pixel value or less) areextracted from the primary candidates of the first darkness, thusyielding first secondary candidates.

At step S310, the image processing control section 206 narrows down theplural primary candidates for posterior pole portion NPAs to onlyregions that follow the course of a blood vessel. More specifically,first the image processing control section 206 (1) extracts bloodvessels. The blood vessels are extracted based on the pixel values usinga method such as morphological processing or binarization. Note that theregions extracted thereby are referred to as vascular regions. Then theimage processing control section 206 uses (2) a method such as distanceconversion to compute a distance between such vascular regions and theperipheral edges of the single or plural primary candidates forposterior pole portion NPAs, or of respective region groups of candidategroups for posterior pole portion NPAs, and selects regions in whichthis computed distance is within a fixed range.

The fixed range referred to here is a first range that is larger than afirst prescribed distance, but smaller than a second prescribed distancelarger than the first prescribed distance (namely, when following bloodvessels).

Thus at step S310, from the primary candidates, the image processingcontrol section 206 extracts as second secondary candidates any regionsfor which the distance to a blood vessel is a first distance or lower.Note that for the second secondary candidates, a region that is a fixedrange away from a blood vessel terminal end may be employed as a secondsecondary candidate.

Either out of step S308 or step S310 may be executed first, oralternatively step S308 and step S310 may be executed simultaneously.The image processing control section 206 executes the image processingillustrated in step S312 after the processing of step S308 and step S310have been completed.

At step S312, the image processing control section 206 performsconsolidation processing to consolidate the first secondary candidatesand the second secondary candidates. Specifically, regions correspondingto both a first secondary candidate (the plural dark regions) and asecond secondary candidate (the plural regions following blood vessels)are extracted, and these regions are identified as posterior poleportion NPAs.

FIGS. 10 are schematic diagrams illustrating part of an FA image inclose up in order to illustrate results of the processing of step S306to step 312 in simplified form. FIG. 10A illustrates blood vessels 400and four primary candidates 406, 408, 410, 412 for posterior poleportion NPAs following the processing of step S306. For ease ofexplanation, the blood vessels 400 are illustrated by black lines inFIG. 10. In FIG. 10B, primary candidates 406A, 408A, 410A illustrated bysolid lines are examples of primary candidates estimated by theprocessing of step S308 and step S310. Moreover, in FIG. 10B, theprimary candidate 412 illustrated with dotted lines is an example of aprimary candidate that has been excluded. In FIG. 10C, estimatedcandidates 406NPA, 408NPA, 410NPA narrowed down as secondary candidatesfrom the primary candidates that correspond to both the first secondarycandidates and the second secondary candidate are illustrated asidentified posterior pole portion NPAs.

Next, at step S314 of FIG. 9, the image processing control section 206extracts the outlines of the posterior pole portion NPAs in the FAimage. The outlines of the posterior pole portion NPAs are images to bedisplayed superimposed on the FA image so as to be allow the positionsof the posterior pole portion NPAs on the FA image to be easilyrecognized. The image processing relating to the posterior pole portionNPAs is performed as described above.

Next, explanation follows regarding processing to pick out misdetectionof the posterior pole portion NPAs at step S108 illustrated in FIG. 7.

The regions detected as the posterior pole portion NPAs as describedabove (step S106) may include regions that do not actually correspond toa posterior pole portion NPA (hereafter referred to as a non-NPAs).Examples of such misdetected non-NPAs include regions corresponding tospots and the like where imaging light does not reach the retina due tothe presence of a photocoagulation spot, soft exudate, or cataracts.

At step S108 of FIG. 7, the image processing control section 206accordingly picks out any misdetected posterior pole portion NPAspresent among the posterior pole portion NPAs detected at step S106.More specifically, the following discrimination methods may be employedto discriminate any misdetected posterior pole portion NPAs presentamong the detected posterior pole portion NPAs.

In a first discrimination method, image processing is performed using animage filter for detecting photocoagulation spots by employing imagedata of known photocoagulation spots. Posterior pole portion NPAs andphotocoagulation spots are discriminated, and if a posterior poleportion NPA is discriminated as being a photocoagulation spot in thisdiscrimination, this posterior pole portion NPA discriminated as being aphotocoagulation spot is picked out as a non-NPA. Note that the imagefilter employed in the first discrimination method is not limited to afilter for detecting photocoagulation spots, and an image filter fordetecting regions such as spots where imaging light does not reach theretina due to the presence of a soft exudate or cataract may be employedinstead.

In a second discrimination method, posterior pole portion NPAs andphotocoagulation spots are discriminated using a machine learning AItrained with image data of known photocoagulation spots. If a posteriorpole portion NPA is discriminated as being a photocoagulation spot, thisposterior pole portion NPA discriminated as being a photocoagulationspot is picked out as a non-NPA. Note that in the second discriminationmethod, there is no limitation to a machine learning AI trained usingimage data of photocoagulation spots, and machine learning training maybe performed using image data illustrating regions such as spots whereimaging light does not reach the retina due to the presence of a softexudate or cataract instead.

In a third discrimination method, data representing non-NPAs is acquiredfrom another image handling device, and any non-NPAs among the detectedposterior pole portion NPAs are picked out using the acquired data.

For example, in the case of soft exudates, the positions of identifiedsoft exudates may be identified using OCT B-scan images. Any posteriorpole portion NPAs at these identified positions are then excluded asbeing soft exudates. Moreover, in the case of photocoagulation spots,positions where laser illumination has been performed with the lasertreatment device 120 may be identified. Any posterior pole portion NPAsat these identified laser-illuminated positions are then excluded asbeing photocoagulation spots. Data representing non-NPAs from anotherimage handling device as employed in this third discrimination method isan example of data for identifying non-NPAs, but there is no limitationthereto. Any data capable of identifying non-NPAs may be employed assuch data.

Detected posterior pole portion NPAs that are misdetected posterior poleportion NPAs are picked out in this manner, enabling the precision ofdetecting posterior pole portion NPAs to be raised.

Moreover, eliminating misdetected posterior pole portion NPAs enablesthe extraction of only posterior pole portion NPAs that are actual nonperfusion areas requiring photocoagulation treatment.

Next, explanation follows regarding UWF-NPA image processing, withreference to FIG. 11. UWF-NPAs are regions where no retinal bloodvessels are present at positions at the peripheral portion of thefundus.

As illustrated in FIG. 11, at step S400 the image processing controlsection 206 performs image emphasis processing on the acquired FA imageto emphasize the vascular portions thereof. This is processing to makethe blood vessels, including capillaries, more prominent in order topredict UWF-NPAs with good precision.

At step S402 the image processing control section 206 performs bloodvessel binarization processing to binarize the emphasized vascular imagein which vascular portions have been emphasized. At step S404, the imageprocessing control section 206 performs distance image creationprocessing to create a distance image using the binary vascular imageresulting from binarizing the emphasized vascular image in which thevascular portions have been emphasized. The distance image is an imagein which the brightness becomes greater as the distance from the edge ofa line segment in the binary image (corresponding to a vascular portion)increases.

At step S406, the image processing control section 206 performsbinarization processing to binarize the distance image. The binarizationprocessing of the distance image performed here results in the distanceimage being binarized to give a binarized distance image in the fundusperipheral portion, this means that the fundus peripheral portion isperipheral of the fundus posterior pole portion, are converted intowhite regions (in some cases white regions also remain at parts of theposterior pole portion).

At step S408, the image processing control section 206 performsprocessing to remove regions containing a predetermined fixed number ofpixels or fewer. This processing is processing performed on thebinarized distance image resulting from binarizing the distance image toconvert white regions that are regions containing the fixed number ofpixels or fewer into black regions. More specifically, plural whitepixels are extracted from the binarized distance image, and a single orplural regions having a surface area of contiguous white pixels of aprescribed surface area or smaller are converted into regions ofcontiguous black pixels. The binarized distance image includes whiteregions representing vascular portions at the fundus posterior poleportion, and at the fundus peripheral portion peripheral to the fundusposterior pole portion. Predetermining the size, for example the numberof pixels in a width direction, of vascular portions at the fundusposterior pole portion as a fixed number of pixels enables the vascularportions at the fundus posterior pole portion to be converted into blackregions in the binarized distance image. Accordingly, white regions atthe fundus peripheral portion accordingly remain in the binarizeddistance image.

At step S410, the image processing control section 206 extracts theoutlines of the UNNT-NPAs by extracting the outlines of any whiteregions remaining in the binarized distance image after removal of thewhite regions containing the fixed number of pixels or fewer. Theoutlines of the UWF-NPAs are images to be displayed superimposed on theFA image so as to enable the positions of the UWF-NPAs in the FA imageto be easily recognized.

The image processing relating to UWF-NPAs is performed in the abovemanner.

Next, a display method for the non perfusion areas (NPAs) will bedescribed in detail with reference to the screen 500 of the display 156of the image viewer 150 illustrated in FIG. 12 and FIG. 13.

FIG. 12 and FIG. 13 are examples of images relating to fundus images ofthe examined eye 12 displayed by executing the “image display method” oftechnology disclosed herein.

When a patient ID is input by an operator performing a fundus imageexamination, the image viewer 150 issues a command to the managementserver 140 to output patient information. The management server 140reads the patient information associated with the patient ID. Themanagement server 140 then reads a FA image, and performs imageprocessing thereon as explained with reference to FIG. 7. The outlineimage of the fundus region, the outline images of any posterior eyeportion NPAs, the outline images of any UWF-NPAs, and the outline imagesof any misdetected posterior pole portion NPAs corresponding to theobtained FA image are then saved in the storage device 154 of themanagement server. Moreover, an image is created in which the outlineimage of the fundus region subjected to the image processing, outlineimages of any posterior pole portion NPAs, outline images of anyUWF-NPAs, and outline images of any misdetected posterior pole portionNPAs are superimposed on the FA image, a display screen for use on theimage viewer 150 is generated, and image data for this display screen issent to the image viewer 150. The display control section 104 of theimage viewer 150 performs control to display the image represented bythe display screen image data from the management server 140 on thedisplay 156.

When an outline image display state, described later, is instructed bythe operator, the image viewer 150 generates a display screen inresponse to the instruction, and issues a command to the managementserver 140 to output image data for the display screen. The processingsection 108 of the image viewer 150 receives the image data for thedisplay screen corresponding to the instructed outline image displaystate, and the display control section 104 performs control to displaythis image on the display 156.

In the example illustrated in FIG. 12, display contents relating to nonperfusion areas (NPAs) are displayed on the screen 500 on the display156 of the image viewer 150. The screen 500 includes a patientinformation display field 502 a fundus image display field 504, and anoption instruction display field 506.

The patient information display field 502 includes a patient ID displayfield 502A, a patient name display field 502B, an age display field502C, a target eyeball display field 502D, an imaging date/time displayfield 502E, an eye axial length display field 502F, and a visual acuitydisplay field 502G. The image viewer 150 acquires the patient ID, thepatient name, the age, the target eyeball, the imaging date/time, theeye axial length, and the visual acuity stored in the management server140. The image viewer 150 respectively displays the acquired patient ID,patient name, age, target eyeball, imaging date/time, eye axial length,and visual acuity in the patient ID display field 502A, the patient namedisplay field 502B, the target eyeball display field 502D, the imagingdate/time display field 502E, the eye axial length display field 502F,and the visual acuity display field 502G.

The image viewer 150 displays the FA image corresponding to the patientin the fundus image display field 504 of the screen 500. Moreover, theimage viewer 150 displays the outline images of the non perfusion areas(NPA) superimposed on the FA image in order to facilitate observationand diagnosis of the fundus of the examined eye 12.

The example illustrated in FIG. 12 illustrates a display state of afundus region outline image 504A on the FA image, posterior eye portionNPA outline images 504B, 504C, and a UWF-NPA outline image 504D aredisplayed. The example illustrated in FIG. 13 illustrates a displaystate in which the fundus region outline image 504A is not displayed onthe fundus region of the FA image, but the posterior eye portion NPAoutline images 504B, 504C, the UWF-NPA outline image 504D, and amisdetected posterior pole portion NPA are displayed.

The operator of the image viewer 150 (a doctor, for example) may desireto change the display state of the images being displayed in the fundusimage display field 504. To do this, the option instruction displayfield 506 includes instruction buttons for selecting instructionsregarding the display state of the images displayed in the fundus imagedisplay field 504, and includes display fields to display instructionresults.

In the example illustrated in FIG. 12, the option instruction displayfield 506 includes a pull-down style instruction button 506A to instructwhether or not to display the outline image of the fundus region on theFA image and instruct a display color thereof, and a display field 506Bto display the instruction result. The display state of the outlineimage of the fundus region displayed in the fundus image display field504 may be changed by a selection instruction of the instruction button506A by the operator of the image viewer 150 (a doctor, for example).For example, whether or not to display the outline image of the fundusregion and the display color thereof is instructed by the operator bymanipulating an input means such as a mouse 155M. When whether or not todisplay the outline image of the fundus region and the display colorthereof have been instructed, the image viewer 150 displays the outlineimage of the fundus region as per the instructed display/non-display anddisplay color.

The option instruction display field 506 includes a pull-down styleinstruction button 506C to instruct whether or not to display theoutline images of posterior pole portion NPAs and instruct a displaycolor thereof, and a display field 506D to display the instructionresult. The display state of the outline images of the posterior poleportion NPAs displayed in the fundus image display field 504 may bechanged by a selection instruction performed by the operator of theimage viewer 150 using the instruction button 506C. For example, whetheror not to display the outline images of the posterior pole portion NPAsand the display color thereof are instructed by the operatormanipulating an input means such as the mouse 155M. When whether or notto display the outline images of the posterior pole portion NPAs and thedisplay color thereof have been instructed, the image viewer 150displays the outline images of the posterior pole portion NPAs as perthe instructed display/non-display and display color.

Moreover, the option instruction display field 506 also includes apull-down style instruction button 506E to instruct whether or not todisplay the outline images of UWF-NPAs and instruct a display colorthereof, and a display field 506F to display the instruction result. Thedisplay state of the outline images of the UWF-NPAs displayed in thefundus image display field 504 may be changed by the operator of theimage viewer 150 by a selection instruction using the instruction button506E. For example, whether or not to display the outline images of theUWF-NPAs and the display color thereof are instructed by the operatormanipulating an input means such as the mouse 155M. When whether or notto display the outline images of the UWF-NPAs and the display colorthereof have been instructed, the image viewer 150 displays the outlineimages of the UWF-NPAs as per the instructed display/non-display anddisplay color.

Moreover, the option instruction display field 506 also includes apull-down style instruction button 506G to instruct whether or not todisplay the outline images of misdetected posterior pole portion NPAs,which are regions having a low likelihood of being a posterior poleportion NPA despite being detected as posterior pole portion NPAs, andinstruct a display color thereof, and a display field 506H to displaythe instruction result. The display state of the outline images of themisdetected posterior pole portion NPAs displayed in the fundus imagedisplay field 504 may be changed by the operator of the image viewer 150by a selection instruction using the instruction button 506G. Forexample, whether or not to display the outline images of the misdetectedposterior pole portion NPAs and the display color thereof are instructedby the operator manipulating an input means such as the mouse 155M. Whenwhether or not to display the outline images of the misdetectedposterior pole portion NPAs and the display color thereof have beeninstructed, the image viewer 150 displays the outline images of themisdetected posterior pole portion NPAs as per the instructeddisplay/non-display and display color.

The image viewer 150 acquires from the management server 140 the FAimage, the fundus region outline images, the posterior eye portion NPAoutline images, the UWF-NPA outline images, and the misdetectedposterior pole portion NPA outline images. Then, from out of the fundusregion outline image, the posterior eye portion NPA outline images, theUWF-NPA outline images, and the misdetected posterior pole portion NPAoutline images, those outline images instructed by the operator aredisplayed superimposed on the FA image. Note that generation of thedisplay image to display outline images such as the non perfusion areas(NPA) superimposed on the FA image may be performed either in themanagement server 140 or in the image viewer 150.

Note that display states of outline images displayed in the fundus imagedisplay field 504 of the screen 500 by the image viewer 150 are notlimited to the display states illustrated in FIG. 12 and FIG. 13. Forexample, various changes may be made to the line thicknesses and linetypes of the outline images, for example by employing dotted lines andsolid lines. Accordingly, the operator may adopt different colors forthe respective display colors of the outline images, or may change theline types for the outline images of the fundus region, the posteriorpole portion NPAs, the UWF-NPAs, and the misdetected posterior poleportion NPAs, such that the differences can be seen. Changes may also bemade to both the colors and line types of the outline images.

The fundus image display field 504 of the screen 500 is displayed by theimage viewer 150, and the posterior pole portion NPAs identified by theoutline images of the posterior pole portion NPAs, are one type ofinformation allowing a doctor to diagnose diabetic retinopathy, diabeticmacular edema, retinal vein occlusion, or the like, and to determine orcheck on pathological progression or results of treatment.

Moreover, the UWF-NPAs identified by the UWF-NPA outline images are onetype of information allowing a doctor to confirm an early diagnosis. Forexample, such information may assist an ophthalmologist in diagnosingdiseases that initially manifest in the peripheral portion of thefundus, such as “pre-proliferative diabetic retinopathy”, “proliferativediabetic retinopathy”, “branch retinal vein occlusion”, “Coats'disease”, “non-infectious uveitis”, and the like, and symptoms for whichexamination of the fundus is useful when finalizing a diagnosis.

Furthermore, the detection of posterior pole portion NPAs may assistdetermination of the results of treatment such as photocoagulation, anddetermination as to whether or not additional surgery is required.

The visualization of UWF-NPAs may assist in early diagnosis andestablishing the state of progression of diabetic retinopathy and thelike, and enable quantitative results of drug treatments such as withvascular endothelial growth factor (VEGF) and blood pressure control tobe ascertained.

Explanation follows regarding various modified examples of thetechnology disclosed herein.

FIRST MODIFIED EXAMPLE

Although in the above exemplary embodiment a case has been described inwhich the image processing control section 206 executes posterior poleportion NPA image processing (step S106 and step S108), and UWF-NPAimage processing (step S110), the technology disclosed herein is notlimited thereto. For example, as illustrated in FIG. 14, the imageprocessing control section 206 may execute the UWF-NPA image processingalone. The processing illustrated in FIG. 14 is similar to thatdescribed above, and detailed explanation thereof will be omitted.

Note that the first modified example includes the following technicalcontent.

(1) An image processing method including a step of acquiring a fundusimage, and a step of extracting a non perfusion area in a regionincluding a periphery of a fundus region from the fundus image.

(2) An image display method including a step of acquiring a fundus imageand information relating to a non perfusion area in a region including aperiphery of a fundus region extracted from the fundus image, and a stepof displaying the non perfusion area superimposed on the fundus image.

(3) An image processing program that causes a computer to execute theimage processing method described at (1).

(4) An image display program that causes a computer to execute the imagedisplay method described at (2).

(5) An image processing device including a fundus image acquisitionsection configured to acquire a fundus image, and a non perfusion areaextraction section configured to extract from the fundus image a nonperfusion area in a region including a periphery of a fundus region.

(6) An image display device including an acquisition section configuredto acquire a fundus image and information relating to a non perfusionarea in a region including a periphery of a fundus region extracted fromthe fundus image, and a display section configured to display the nonperfusion area superimposed on the fundus image.

SECOND MODIFIED EXAMPLE

Although in the above exemplary embodiment the image processing controlsection 206 executes the posterior pole portion NPA image processing andthe 1LTWF-NPA image processing, and in the first modified example theimage processing control section 206 executes the UWF-NPA imageprocessing alone, the technology disclosed herein is not limitedthereto. For example, as illustrated in FIG. 15, the image processingcontrol section 206 may execute the posterior pole portion NPA imageprocessing alone. The processing illustrated in FIG. 15 is similar tothat described above and so detailed explanation thereof will beomitted.

Note that the second modified example includes the following technicalcontent.

(7) An image processing method including a step of acquiring a fundusimage, and a step of extracting a non perfusion area in a regionincluding a center of a fundus region from the fundus image.

(8) The image processing method described at (7), further including astep of extracting a region likely to have been misdetected from out ofthe extracted non perfusion areas, and a step of excluding the regionlikely to have been misdetected from the extracted first non perfusionareas.

(9) An image display method including a step of acquiring a fundus imageand information relating to a non perfusion area in a region including acenter of a fundus region extracted from the fundus image, and a step ofdisplaying the non perfusion area superimposed on the fundus image.

(10) An image processing program that causes a computer to execute theimage processing method described at (7) or (8).

(11) An image display program that causes a computer to execute theimage display method described at (9).

(12) An image processing device including a fundus image acquisitionsection configured to acquire a fundus image, and a non perfusion areaextraction section configured to extract from the fundus image a nonperfusion area in a region including a posterior pole portion of afundus region.

(13) An image display device including an acquisition section configuredto acquire a fundus image and information relating to a non perfusionarea a non perfusion area in a region including a posterior pole portionof a fundus region extracted from the fundus image, and a displaysection configured to display the non perfusion area superimposed on thefundus image.

THIRD MODIFIED EXAMPLE

Although in the exemplary embodiment described above the managementserver 140 executes the image processing program illustrated in FIG. 7in advance, the technology disclosed herein is not limited thereto. Aconfiguration may be adopted in which the image viewer 150 transmits animage processing command to the management server 140, and themanagement server 140 executes the image processing program of FIG. 6 inresponse to this command.

FOURTH MODIFIED EXAMPLE

In the exemplary embodiment described above, explanation has been givenregarding examples in which a fundus image having an internalillumination angle of approximately 200° is acquired by the ophthalmicdevice 110. The technology disclosed herein is not limited thereto, andthe technology disclosed herein may also be applied in a configurationin which a fundus image having an internal illumination angle of 100° orless is captured by an ophthalmic device, or in a configuration in whicha montage image synthesized from plural fundus images is employed.

FIFTH MODIFIED EXAMPLE

Although in the exemplary embodiment described above the fundus image iscaptured by the ophthalmic device 110 provided with an SLO imaging unit,the technology disclosed herein may also be applied to a configurationin which an image obtained by OCT angiography is employed.

SIXTH MODIFIED EXAMPLE

In the exemplary embodiment described above, the management server 140executes the image processing program. The technology disclosed hereinis not limited thereto. For example, the ophthalmic device 110 or theimage viewer 150 may execute the image processing program.

SEVENTH MODIFIED EXAMPLE

Although explanation has been given in the exemplary embodimentdescribed above regarding an example in which the ophthalmic system 100is provided with the ophthalmic device 110, the laser treatment device120, the management server 140, and the image viewer 150, the technologydisclosed herein is not limited thereto. For example, as a firstexample, a configuration may be adopted in which the laser treatmentdevice 120 is omitted and the ophthalmic device 110 further incorporatesthe functionality of the laser treatment device 120. Alternatively, as asecond example, a configuration may be adopted in which the ophthalmicdevice 110 further incorporates the functionality of at least one out ofthe management server 140 or the image viewer 150. For example, themanagement server 140 may be omitted in cases in which the ophthalmicdevice 110 includes the functionality of the management server 140. Insuch cases, the image processing program is executed by the ophthalmicdevice 110 or the image viewer 150. Alternatively, the image viewer 150may be omitted in cases in which the ophthalmic device 110 includes thefunctionality of the image viewer 150. As a third example, aconfiguration may be adopted in which the management server 140 isomitted, and the image viewer 150 executes the functionality of themanagement server 140.

EIGHTH MODIFIED EXAMPLE

Although in the above exemplary embodiment explanation has been givenregarding a case in which posterior pole portion NPAs are detected bythe image processing control section 206 executing the processingillustrated in FIG. 9, technology disclosed herein is not limitedthereto. For example, the posterior pole portion NPAs may be detected bythe processing illustrated in FIG. 16.

Explanation follows regarding such posterior pole portion NPA imageprocessing, with reference to FIG. 16.

As illustrated in FIG. 16, at step S1304, processing is performed on theacquired FA image to remove the retinal blood vessels. A Gaussian filteris then employed to remove high frequency components from the image fromwhich the retinal blood vessels have been processed out to create a lowfrequency component image.

The image processing control section 206 then, at step S1306, performsbrightness compensation on the peripheral portion of the fundus bysubtracting the low frequency component image from the FA image.

The image processing control section 206 then performs NPA detectionprocessing at step S1308. More specifically, plural pixels having afirst darkness or lower are extracted from the fundus image obtained atstep S1306 and having brightness-compensated fundus peripheral portions,and a single or plural regions having a surface area of contiguous darkpixels of the first darkness or lower of a prescribed surface area orlarger are detected as posterior pole portion NPAs. Then at step S1310,the image processing control section 206 extracts outlines of thedetected posterior pole portion NPAs, and the processing is then ended.

As described above, in the eighth modified example, image processingrelating to posterior pole portion NPAs is performed. In contrast to theposterior pole portion NPA processing illustrated in FIG. 9, there is noneed to perform the extraction processing of step S308 and step S310,thereby enabling faster processing and thus enabling the number offundus images processed per unit time to be increased.

NINTH MODIFIED EXAMPLE

Although the exemplary embodiment described above is configured suchthat the image 500 displayed on the image viewer 150 includes theoutline image of the fundus region, the outline images of posterior eyeportion NPAs, the outline images of UWF-NPAs, and the outline images ofmisdetected posterior pole portion NPAs superimposed on the FA image,the technology disclosed herein is not limited thereto. For example, aconfiguration may be adopted in which the image displayed includes theoutline image of the fundus region, the outline images of posterior eyeportion NPAs, the outline images of UWF-NPAs, and the outline images ofmisdetected posterior pole portion NPAs superimposed on a face-on image(en-face image) obtained from a color fundus image and OCT data. In suchcases, positional alignment is performed between the FA image and theother image, such as the color fundus image, and the outline images arethen displayed superimposed at appropriate positions on the color fundusfundus image.

OTHER MODIFIED EXAMPLES

The data processing as explained in the exemplary embodiment describedabove is merely examplary. Obviously, unnecessary steps may be omitted,new steps may be added, or the processing sequence may be rearrangedwithin a range not departing from the spirit of the present disclosure.

Although explanation has been given in the exemplary embodimentdescribed above regarding an example in which a computer is employed toimplement data processing using a software configuration, the technologydisclosed herein is not limited thereto. For example, instead of asoftware configuration employing a computer, the data processing may beexecuted solely by a hardware configuration such as a field programmablegate array (FPGA) or an application specific integrated circuit (ASIC).Alternatively, a configuration may be adopted in which some processingout of the data processing is executed by a software configuration, andthe remaining processing is executed by a hardware configuration.Explanation of the Reference Numerals

100 ophthalmic system

110 ophthalmic device

120 laser treatment device

140 management server

150 image viewer

204 display control section

206 image processing control section

208 processing section

262 CPU

254 storage device

1. An image processing method comprising: acquiring a fundus image; extracting a first non perfusion area in a first region of a fundus from the fundus image, by a first image processing; and extracting a second non perfusion area in a second region, which is a peripherical region of the fundus, from the fundus image, by a second image processing.
 2. The image processing method of claim 1, further comprising extracting a fundus region from the fundus image.
 3. The image processing method of claim 2, wherein the first region is a posterior pole portion of the fundus region, and the second region is a peripheral portion of the fundus region.
 4. The image processing method of claim 1, further comprising: extracting a third non perfusion area likely to have been misdetected among extracted first non perfusion areas; and excluding the third non perfusion area from the extracted first non perfusion areas.
 5. The image processing method of claim 1, wherein the fundus image is a fluoroscopic fundus image or an OCT image acquired by optical coherence tomography.
 6. An image display method comprising: acquiring a fundus image, a first non perfusion area in a first region of a fundus extracted from the fundus image by a first image processing, and a second non perfusion area in a second region of the fundus extracted from the fundus image by a second image processing; and displaying at least one of the first non perfusion area or the second non perfusion area superimposed on the fundus image.
 7. The image display method of claim 6, further comprising displaying a fundus region superimposed on the fundus image.
 8. The image display method of claim 6, further comprising displaying a selection screen for selecting a type of non perfusion area to superimpose on the fundus image.
 9. A storage medium being not transitory signal and stored with an image processing program executable by a computer to perform the image processing method of claim
 1. 10. A storage medium being not transitory signal and stored with an image display program executable by a computer to perform the image display method of claim
 6. 11. An image processing device comprising: a fundus image acquisition section configured to acquire a fundus image; a first non perfusion area extraction section configured to extract, a first non perfusion area in a first region of a fundus from the fundus image, by a first image processing; and a second non perfusion area extraction section configured to extract, a second non perfusion area in a second region of the fundus from the fundus image, by a second image processing.
 12. An image display device comprising: an acquisition section configured to acquire a fundus image, a first non perfusion area in a first region of a fundus extracted from the fundus image by a first image processing, and a second non perfusion area in a second region of the fundus from the fundus image, by a second image processing; and a display section configured to display at least one of the first non perfusion area or the second non perfusion area superimposed on the fundus image. 